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BACKGROUND OF THE INVENTION 
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The present invention relates to audio visual information systems, and more 



specifically to a system for describing , classifying, and retrieving audiovisual information for 

compact descriptions of relationships. 

The amount of multimedia content available on the World Wide Web and in 

numerous other databases is growing out of control. However, the enthusiasm for developing 
1 0 multimedia content has led to increasing difficulties in managing accessing and identifying 

and such content mostly due to their volume. Further more, complexity and a lack of 

adequate indexing standards are problematic. To address this problem, MPEG-7 is being 

developed by the Moving Pictures Expert Group (MPEG) , which is a working group of 

ISO/IEC. In contrast to preceding MPEG standards such as MPEG-1 and MPEG-2 which 
1 5 relate to coded representation of audio-visual content, MPEG-7 is directed to representing 

information relating to content, and not the content itself. 

The MPEG-7 standard, formally called the "Multimedia Content Description 

Interface" seeksfto tc/provide a rich set of standardized tools for describing multimedia 

content. It is the objective to provide a single standard for providing interoperable, simple 
20 and flexible solutions to the aforementioned problems vis-a-vis indexing, searching and 

retrieving multimedia content. Software and hardware systems for efficiently generating and 

interpreting MPEG-7 descriptions are being developed. 

More specifically, MPEG-7 defines and standardizes the following: (l) a 

core set of Descriptors (Ds)for describing the various features of multimedia content; (2) 
25 Description Schemes (DSs) which are pre-defined structures of Descriptors and their 

relationships; and (3) a Description Definition Language (DDL) for defining Description 

Schemes and Descriptors. 

A Descriptor (D) defines both the semantics and the syntax for representing a 

particular feature of audiovisual content. A feature is a distinctive characteristic of the data 
30 which is of significance to a user. 

As noted, DSs are pre-defined structures of Descriptors and their relationships. 

Specifically, the DS sets forth the structure and semantics of the relationships between its 

components having either Descriptors and/or Description Schemes. To describe audiovisual 



content, a concept known as syntactic structure which specifies the physical and logical 
structure of audiovisual content is utilized. 

The Description Definition Language (DDL) is the language that allows the 
creation of new Description Schemes and Descriptors. It also allows the extension and 
5 modification of existing Description Schemes. The DDL has to be able to express spatial, 
modification of existing Description Schemes. The DDL has to be able to express spatial, 
temporal, structural, and conceptual relationships between the elements of a DS, and between 
DSs. 

Conventional systems have been unable to address the issue of weights for 
10 description schemes. A "weight" is a method of establishing the strength of a relationship 
between description schemes, representing properties or parts of a description. 
Disadvantageously, conventional systems place weights on entities themselves so that future 
changes to weights cannot be carried out in a simplified manner. 

Therefore there is a need to resolve the aforementioned disadvantage and the 
1 5 present invention meets this need. 

SUMMARY OF THE INVENTION 
A method permitting compact ways to update relationships between entities in 

an audiovideo sequence, or serial set of sequences. The method comprises (1) writing a 
20 description between the entities, the description containing relations; (2) determining the 

relations that may be represented by parameters, each parameter having a numerical value; 

and (3) obtaining from the user, one or more of the following: (a) the numerical value for the 

parameter; (b) a description of the parameter containing the numerical value; and (c) a 

description capable of setting the parameter dynamically. 
25 According to another aspect of the present invention, the method further 

comprises combining a State DS with an additional field in a GraphType DS. 

According to another aspect of the present invention, combining allows a set 

of parameters to determine the strength of an edge, seen as a fuzzy member of the relation 

defined by edges on a set of vertices. 
30 According to another aspect of the present invention, the method further 

comprises running, by a user, a query based on membership in the relation. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram of a technique for using the present invention. 
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Fig. 2 is block diagram of an alternate technique for using the present 



invention. 



Fig. 3 is block diagram of an alternate technique for using the present 



invention. 
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DETAILED DESCRIPTION OF THE INVENTION 
The most concise method of offering a measure of the strength of a 



relationship, comes from examining the graph as a fundamental object. A graph is a set E of 



10 vertex for each edge. If this mapping is 1-1, then it the mapping specifies a subset of Fx F, if 
not, then it can be broken into a set of partial maps E -> Fx F, which form subsets of Fx F 
each on some subset of the set of edges. If n-ary relations are examined, the proper subsets 
of the proper Cartesian product space may be mapped, that is, the product space of the n 
spaces involved in the relation. 

1 5 The simplest case considered here is as follows. The form of E is a relation on 

the set F of vertices. As such the above mapping may be viewed as an inclusion, to produce 
EczVxV. Giving E as a subset, allows the proper framework to evaluate measures of 
strength, by measuring the degree to which a particular edge is a member of E. This 
establishes a membership function the: Fx F, giving both a way of specifying strength or 

20 confidence in a relationship, and a simple mechanism for achieving it. 0 This definition of a 
fuzzy relation differs from the conventional definitions which defines a relationship on the 
whole Cartesian product, whereas the present invention defines fuzzy relationship on a subset 
This is of great advantage in writing concise descriptions, since we use the lack of a relation 
to abbreviate the Graph written, and this is formally equivalent to the relation having the 

25 membership function value of zero. It also preserves the ability to make Boolean queries on 
the structure of the graph, where otherwise we are limited to complete graphs (graphs with 
edges between each n nodes). The definition is beneficial in parametrized membership 
functions below, because the parametrization function is then an implicit function. 



30 membership function is defined for the graph relation itself in one step. In practical terms 

this involves adding a weight attribute to each Edge in a graph, or Link outside a graph. The 
problem with this is that all of the mechanisms for updating the value of an edge in an 
interactive or streaming environment is now implicit. It is possible to have the calculations 
for the membership function be more explicit, by using the State DS. 



cadges, a set Fof vertices, together with a mapping E — > Fx F specifying the start and end 



The very simplest technique for "fuzzifying" graphs is to allow that the 
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In reality, the relation E on Vx Vis part of a decomposition involving entities 
which are represented by the vertices in K, and the various and sundry relationships in which 
they are involved. This decomposition maps the description on to all the relations of which 
any part participates. When a relationship is written in Graph DS, a relation R c A x B, is 
5 being formed, after which R is mapped into £asa subset. The values which determine the 
membership function value in E are therefore parameters affecting the entity types A and B, 
and the relation R which maps into the graph. 

This suggests use of a parametrized membership function. This means that if 
R is a fuzzy set, and x is an element, we split the mapping via a parameter space, PS, that is, 

10 defining m R (x) by defining/ :AxB^> PS,g : PS , and m R {x) = g ° f(x). How this works is 
that we define, the relation R, as a "relation type". In the relation type, we define the 
parameter function g, leaving the parametrization function / to be defined implicitly when 
edges are defined. Suppose, for an example, that we have g(u + v + t + w) = (u +v+H-w)/4, 
w,v,£w reals(double)constrained. Define a graph with an Edge(a,b), and a State DS with 

15 attribute-value pairs these 4 variables, and a Link to Edge(a,b) (or from Edge(a,b) if the 
scoping is easier). 

As the AV sequence progresses through time, the values of these variables are 
updated and this changes the confidence in this edge. The dependencies of the membership 
function on parameters are expressed in the membership function declaration. Since these are 
20 attribute value pairs, it is also possible to link to the attributes in the two or more nodes 

participating in the relation instance. This circumstance is likely to be common, it asserts that 
the participants in a relation determine the level of that relation. 

£ >^^ Figs. 3 above of the ripe fruit illusto&tes this usage. The use of this is for 
"running descriptions", like live feeds. Fig. 1 is ^block diagram of a technique for using the 
25 present invention. In Fig. 1 , a member function Expressed in the Relation DS is shown. 

tA^Fig. 2 is block diadf am of an alternate technique for using the present 
invention. In Fig. 2, a membej/runction expressed by a SemanticStateDS is shown. 





y 0 ^ Concretely, define each as the "temperature" of 4 romantic relationships a 

certain unfaithful husband in the soap opera "The Coming Storm" is engaged in. Let 
30 Edgel(a,b) be an edge in the graph of the relations between all characters in "The Coming 
Storm", a is the husband, b is his wife, and Edgel is an element of the relation (i.e. an 
instance of the relation type) "isn't honest with". The state is the only element in this simple 
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scenario that needs to be updated to give soap opera fans who missed the show a rundown on 
how the main marriage in "The Coming Storm" is doing. 
Syntax: 

The following gets added to GraphType. 
5 <complexType name=MemberFunction> 

<attribute name="id" type="ID" use="optional" /> 
<attribute name="parameter" type="AttributeValuePair" 
minOccurs="0" maxOccurs-'unbounded" /> 
<attribute name="functionType" type="ControlledTerm" /> 
10 <i£/complexType> 

Therefore, the present invention uses the State DS combined with an 
additional field in GraphType that allows a set of parameters to determine the strength of an 
edge, seen as a fuzzy member of the relation defined by edges on the set of vertices. This is 
shown to allow compact ways to update relationships between entities in an audiovideo 
1 5 sequence, or serial set of sequences. 

While the above is a complete description of exemplary specific embodiments 
of the invention, additional embodiments are also possible. Thus, the above description 
should not be taken as limiting the scope of the invention, which is defined by the appended 
claims along with their full scope of equivalents. 

20 
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